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ABSTRACT 


The  T  .anrling  Craft  Air  Cushion  (LCAC)  is  an  amphibious  hovercraft  that  can  ride  on  a  cushion  of  air  across  land 
or  sea.  Its  control  features  are  similar  to  a  helicopter,  and  it  is  designed  to  transport  weapons,  cargo,  equipment 
and  combat  personnel.  In  the  1980s,  the  LCAC  community  was  experiencing  a  high  attrition  rate,  partially  due  to 
the  absence  of  any  valid  selection  mechanisms  for  crewmembers.  The  Naval  Aerospace  Medical  Research 
Laboratory  (NAMRL)  developed  a  selection  system  for  the  LCAC  operators  and  engineers,  and  the  system  was 
transitioned  to  the  Naval  Aerospace  Medical  Institute,  (now  the  Naval  Operational  Medicine  Institute)  in  1992. 
When  similar  attrition  problems  were  seen  in  the  navigator  community,  we  were  again  tasked  with  developing  a 
selection  system  in  FY  94-95.  Concurrent  validation  of  the  system  was  done  using  58  LCAC  navigators.  The 
preliminary  predictive  model  was  generated,  and  the  cut-off  score  was  derived  from  the  sponsor’s  operational 
manpower  needs.  To  date,  30  candidates  have  been  screened;  25  have  been  recommended  for  training.  Thirteen  of 
these  candidates  have  entered  training  and  completed  the  full  22-week  syllabus.  Five  of  the  13  recommended 
candidates  attrited  dining  training,  with  “failure  to  handle  multitasking”  as  the  main  reason  sited.  The  system  is 
still  being  evaluated  to  (1)  possibly  include  more  multitasking  tests,  (2)  modify  the  predictive  algorithm,  and/or  (3) 
raise  the  cut-off  score  to  further  reduce  the  attrition  rates. 
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INTRODUCTION 


A  Landing  Craft  Air  Cushion  Vehicle,  or  LCAC,  is  one  of  the  most  unusual  transport  vehicles  in  the  world 
(see  Fig.  1).  The  LCAC  is  a  hovercraft  that  rides  on  a  cushion  of  air  across  land  and/or  sea.  It  can  reach  70%  of 
the  world’s  coast  line,  whereas  most  ships  can  only  reach  30%.  The  control  features  of  the  craft  are  similar  to 
those  of  a  helicopter,  with  three  axes  of  rotation:  vertical,  forward/backward,  and  side  to  side.  The  LCAC  was 
specifically  designed  to  transport  weapons,  equipment,  and  combat-ready  personnel  from  ship  to  shore.  In  the 
future,  the  system’s  missions  may  be  expanded  in  the  future  to  include  medical  evacuations  during  wartime 
operations,  noncombatant  evacuations,  and  lane  clearing  during  amphibious  landings.  For  personnel  delivery  and 
medical  evacuations,  the  LCAC  deck  is  modified  to  include  a  Personnel  Transport  Module  (PTM),  which  is  a 
series  of  metal  boxes  that  sit  in  the  center  of  the  deck.  For  lane-clearing  operations,  the  concept  is  to  have  the 
LCAC  fire  several  rounds  of  explosive  off  its  bow  to  detonate  mines  and  clear  a  channel  for  troop  delivery. 


Figure  1.  Landing  Craft  Air  Cushion  Vehicle. 


The  LCAC  crew  consists  of  five  enlisted  personnel:  loadmaster,  deckmaster,  craftmaster,  engineer,  and 
navigator.  The  loadmaster  and  deckmaster  are  responsible  for  the  load  (personnel,  tanks,  etc.)  that  is  being 
transported,  and  they  are  usually  located  in  the  port  crew  stations  during  sea  operations.  The  craftmaster,  engineer, 
and  navigator  are  all  located  in  the  starboard  crew  stations.  The  craftmaster  is  responsible  for  fire  actual  control  of 
the  LCAC  vehicle.  The  engineer  monitors  engine  performance  and  limitations  and  relays  the  information  to  the 
craftmaster.  The  navigator  is  responsible  for  maintaining  and  monitoring  navigation  equipment  as  well  as  plotting 
courses  and  planning  for  collision  avoidance.  Thts  mformation  is  also  relayed  to  the  craftmaster.  In  addition  to 
these  duties,  the  navigators  and  the  engineers  have  some  external  visual  scanning  responsibilities  to  further  support 
the  craftmaster  in  control  of  the  craft.  The  crew  communication  during  actual  operations  is  extensive.  Lastly, 
LCAC  navigators  are  also  responsible  for  maintaining  personnel  and  training  logs  ashore. 

In  the  mid  to  late  1980s,  the  LCAC  operator  and  engineer  community  were  facing  a  training  attiriton  rate  of 
40-60%  (Eakin,  1990).  Due  to  these  excessive  rates,  combined  with  the  relative  high  cost  of  this  training,  the 
Naval  Aerospace  Medical  Research  Laboratory  (NAMRL)  was  funded  by  the  Naval  Sea  Systems  Command 
(NAVSEA)  to  develop  a  selection  system  to  reduce  this  rate.  The  system  was  completed,  and  in  1992  it  was 
transitioned  to  the  Naval  Aerospace  Medical  Institute  (now  the  Naval  Operational  Medicine  Institute,  NOMI) 
(Helton,  Nontasak,  &  Dolgin,  1992).  The  attrition  rate  has  been  reduced  to  approximately  15%  since  that  time 
(Robertson  &  Nontasak,  1996).  In  the  early  1990s  a  similar  problem  developed  within  the  navigator  community 
where  attrition  rates  were  40-50%.  These  high  attritions  were  making  it  difficult  for  the  LCAC  community  to  meet 
their  deployment  needs,  and  NAMRL  was  again  tasked  with  rapidly  developing  a  screening  system  for  the 
navigator  position. 


1 


METHODS 


SUBJECTS 

The  original  model  sample  included  58  LCAC  navigators  from  Assault  Craft  Unit  (ACU)-4,  Little  Creek, 
Virginia,  and  ACU-5,  Camp  Pendleton,  California.  All  navigators  included  in  the  original  validation  of  the  system 
had  completed  the  LCAC  navigators  course  and  were  qualified  for  the  navigator  position.  The  number  of  operating 
hours  as  an  actual  navigator  ranged  from  1 12  to  over  3500  h.  This  group  of  individuals  was  used  as  the  concurrent 
validation  sample.  Thirty  navigators  were  from  ACU-4,  and  28  were  from  ACU-5.  The  subjects  were  exclusively 
male  and  had  achieved  the  grade  of  at  least  a  first  class  petty  officer,  or  chief  (E-6  or  E-7). 

The  subjects  used  for  the  predictive  modeling  consisted  of  enlisted  personnel  (n  =  30)  who  were  interested  in 
entering  the  LCAC  navigator  training  program.  They  are  routinely  sent  to  the  Naval  Operational  Medicine 
Institute  for  physical  screening  for  the  program.  As  part  of  that  screening,  they  are  sent  to  NAMRL  for  testing  on 
the  performance  battery  described  below.  All  but  two  subjects  were  male,  and  all  subjects  had  achieved  a  grade  of 
at  least  a  second  or  first  class  petty  officer  (E-5  or  E-6).  No  subjects  reported  any  previous  experience  as  a 
navigator. 

MATERIALS 

Hardware 

All  testing  was  done  on  an  IBM/PC  compatible  (386/25  MHz)  or  greater,  VGA  monitor,  joystick,  numeric 
keypad,  and  headphones.  The  system  setup  can  be  seen  in  Fig.  2. 


Stereo 

Headphone 


LCAC  Navigator  Selection  System  (LNSS) 

Key  personnel  in  the  LCAC  community  were  interviewed  to  identify-  the  skills  and  abilities  necessary  to  be  a 
successful  crew  member.  Based  on  these  interviews  and  task  rankings,  several  in-house  tests  were  identified  to  be 
included  as  part  of  the  LNSS  (Nontasak,  Dolgin,  &  Griffin,  1989).  The  test  battery  consisted  of  a  series  of 
individual  and  multiple  tasks.  The  subtests  were  as  follows:  Pychomotor  test/Dichotic  Listening  Task  (DLT), 
Absolute  Difference  (AD),  Manikin,  Time  Estimation  Task  (TET),  and  a  Complex  Visual  Task  (CVT). 
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The  psychomotor  test  measures  eye-hand  coordination.  A  joystick  is  located  directly  in  front  of  individuals 
and  moves  in  all  four  directions.  Individuals  are  required  to  maintain  crosshairs  in  the  center  of  the  monitor  (see 
Fig.  3)  for  the  duration  of  the  task.  A  random  drift  function  on  the  crosshairs  forces  subjects  to  make  constant 
adjustments  with  the  joystick  to  maintain  the  crosshairs  at  the  centerpoint.  The  task  is  performed  individually  for 
three  2  minute  trials.  The  tracking  error  obtained  is  the  cumulative  pixel  error  distance  from  the  center  of  the 
screen  across  multiple  trials. 


Figure  3.  Stick. 


The  DLT  is  a  listening  task  in  which  individuals  are  required  to  wear  headphones.  A  string  of  letters  and 
numbers  are  presented  one  at  a  time  via  the  headphones,  however,  the  information  being  heard  in  the  left  ear  is 
different  from  that  in  the  right  ear  (Fig.  4).  Using  the  numeric  keypad,  individuals  respond  to  only  the  numbers 
heard  in  a  particular  ear,  as  designated  by  the  computer.  This  task  is  first  completed  individually  for  12  trials,  and 
then  done  simultaneously  with  the  psychomotor  task  mentioned  above.  The  subjects  are  again  given  a  total  of  a  12 
combined  trials.  The  performance  measure  for  the  stick  task  is  the  logarithm  to  the  base  10  of  the  tracking  error 
score.  The  performance  measure  for  the  DLT  is  a  cumulative  score  for  the  number  correct. 


Sequence 

Left  Ear  Stimulus 

Right  Ear  Stimulus 

Trial# 

“Test  1” 

“Test  1” 

Attention 

“RIGHT’ 

“RIGHT’ 

Stimuli 

Pause 

“R8NSMY2GB7FL6RL5” 

“YL3SR4FZ9XF0FN1L” 

Attention 

“LEFT’ 

“LEFT’ 

Stimuli 

“BF43  79” 

“GL1  5  62” 

Figure  4.  Dichotic  Listening  Task. 

The  AD  task  is  a  cognitive  skills  test.  Single  digit  numbers  are  presented  one  at  a  time  on  the  screen,  and 
individuals  are  asked  to  take  the  absolute  difference  between  the  current  number  and  the  previous  number  on  the 
screen.  inputs  are  made  using  the  numeric  keypad.  For  example,  if  the  first  number  on  the  screen  is  a  4  and  the 
next  number  is  a  3,  then  1  would  be  entered  into  the  keypad.  As  soon  as  that  is  done,  a  new  number  will  appear, 
for  example  a  6.  The  difference  between  the  6,  which  is  the  current  number  and  the  3,  which  was  the  previous 
number  would  be  calculated  and  entered  via  the  keypad.  This  task  is  performed  for  two  2-min  trials.  Performance 
is  measured  as  the  average  number  of  correct  responses  minus  the  number  of  incorrect  responses  over  the  two 
trials.  Reaction  times  for  both  responses  are  also  recorded. 
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The  manikin  is  a  spatial  apperception  or  mental  rotation  task.  An  image  of  a  sailor  is  presented  on  the  screen. 
In  one  hand  he  is  holding  a  green  circle  and  the  other  a  red  square  (Fig.  5).  The  objective  of  the  task  is  to 
determine  which  hand  is  holding  the  red  square.  The  sailor  can  be  in  4  different  positions.  Rightside  up  and 
facing  you,  rightside  up  and  backwards,  upside  down  and  facing  you,  or  upside  down  and  backwards.  Responses 
are  made  using  the  keypad  and  consist  of  a  simple  left  versus  right-hand  determination.  The  test  is  timed  (three  2- 
min  trials),  and  individuals  are  told  to  do  as  many  of  the  manikin  presentations  as  possible,  while  maintaining 
their  accuracy.  Performance  is  measured  as  the  number  of  correct  responses  minus  the  number  of  incorrect 
responses,  as  well  as  the  reaction  time  for  each  correct  and  incorrect  response. 


Figure  5.  Manikin. 


The  TET  (Fig.  6)  is  a  multitask  test  that  measures  time-distance  estimation,  as  well  as  psychomotor  skills.  For 
the  TET,  individuals  are  required  to  perform  two  tasks.  First,  they  are  required  to  keep  a  crosshair  centered  at  the 
bottom  of  the  screen  with  a  left  to  right  motion  of  a  joystick.  During  this  task,  at  the  upper  left  part  of  the  computer 
monitor,  a  horizontally  moving  target  (an  airplane)  enters  the  visual  field  for  about  2  to  3  s  and  then  disappears. 

In  the  upper  right  side  of  the  screen  is  a  target  zone.  From  the  initial  visually  observed  speed  of  the  target,  subjects 
must  determine  when  the  target  will  be  located  directly  above  the  target  zone.  At  that  point,  they  must  squeeze/fire 
the  trigger  button  located  on  the  joystick.  When  they  squeeze  the  trigger  button,  feedback  on  their  accuracy  is  given 
as  a  number  that  appears  over  that  target  zone.  The  number  reflects  the  pixel  error  (distance)  from  the  optimal 
target  zone.  A  positive  number  means  that  they  overestimated  the  speed  of  the  aircraft  and  fired  too  soon. 

Likewise,  a  negative  number  means  that  they  underestimated  the  speed  of  the  aircraft  and  fired  too  late.  The 
smaller  the  number,  the  closer  they  came  to  directly  targeting  the  aircraft.  Between  trials,  the  aircraft  target  can 
travel  at  different  speeds,  however,  within  trials  it  maintains  the  same  speed.  The  number  of  trials  and  speeds  are 
consistent  between  subjects.  Subjects  receive  two  4-min  trials.  Neither  task  is  presented  in  isolation.  Performance 
is  measured  as  the  average  of  the  logarithm  to  the  base  10  of  the  tracking  score  and  the  pixel  error  from  the  target. 
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The  last  subtest  included  in  this  test  battery  is  the  CVT,  which  was  originally  developed  as  a  selection  tool  for 
the  navigator  position  in  the  naval  flight  officer  community  (Morrison,  1988).  The  system  was  designed  to  assess 
an  individual’s  ability  to  encode  verbal  information  and  manipulate  pictorial  displays  in  working  memory. 
Individuals  are  first  presented  a  test  question  (Fig.  7).  The  question’s  content  is  related  to  an  image  (called  the 
tactical  display)  that  will  be  presented  in  the  successive  screen  of  the  test.  An  individual  reads  the  question  and 
then  prompts  the  tactical  display  by  pressing  the  enter  key  on  the  keyboard.  The  display  contains  a  circle  with 
crosshairs  and  a  number  of  objects  of  various  shapes,  sizes,  and  colors.  These  objects  are  located  in  different 
quadrant  locations  and  facing  various  directions  (Fig.  8).  The  goal  of  the  subject  is  to  respond  correctly  to  the 
initial  question  given  the  visual  image  on  the  screen.  Questions  may  be  true  and  false  or  numeric  in  nature.  The 
subject  responds  using  the  keypad.  Once  a  response  is  made,  immediate  feedback  is  given  concerning  whether 
answers  were  correct  or  not.  The  subject  then  must  press  the  enter  key  again  to  proceed  to  the  next  question.  The 
test  contains  60  questions.  Subjects  are  asked  to  complete  all  questions  as  quickly  and  as  accurately  as  possible. 
Performance  is  measured  as  the  number  correct  and  incorrect  responses  out  of  the  total  possible  correct.  In 
addition,  the  time  to  read  the  question;  and  the  reaction  times  for  correct  and  incorrect  questions  are  also  recorded. 
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Question  Number  1 

More  than  7  objects  arc  in  the  right  portion  of  the 
screen  (True  or  False)? 


Figure  7.  Question  Display. 


Figure  8.  Tactical  Display. 


PROCEDURES 

All  subjects  at  ACU-5  and  ACU-4  were  tested  at  their  units  in  November  and  December  1995.  The 
purpose  for  the  project,  and  a  brief  description  of  each  test  was  given,  and  appropriate  informed  consent  procedures 
reviewed.  Performance  measures  on  the  test  were  used  as  the  predictors,  and  instructor  ratings  of  the  navigators 
were  used  as  the  criterion  data.  All  instructors  were  asked  to  rate  individuals  on  their  abilities  as  a  navigator  on  a 
scale  1-10,  with  1  being  optimal.  A  preliminary  predictive  model  was  generated  from  this  concurrent  validation 
sample  and  screening  of  navigator  student  candidates  began  in  May  1996.  For  those  individuals  screened  on  the 
system,  criterion  data  (training  outcome)  were  collected  for  predictive  validation  of  the  system  and  possible 
modification  of  the  model. 
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RESULTS 


CONSTRUCTION  OF  SCORES 

This  initial  section  provides  documentation  on  how  the  eight  scores,  Si,  were  formed.  Four  of  the  scores,  1) 
Percentage  correct  on  CVT,  2)  Reaction  Time  (RT)  to  answers  on  CVT,  3)  RT  to  questions  on  CVT  and,  4) 
Dichotic  Listening  Task  were  not  constructed  from  any  more  basic  variables.  The  other  four  scores,  5)  Absolute 
Difference  Task,  6)  Dual  Task  (Stick  with  DLT),  7)  Manikin,  and  8)  Time  Estimation  Task  were  each  constructed 
from  two  underlying  basic  variables  related  to  the  constructed  score.  All  eight  scores  were  brought  into  a  common 
metric  by  a  process  similar  to  forming  a  standardized  score.  That  is,  a  mean  was  subtracted  from  each  raw  score 
and  then  divided  by  a  standard  deviation  These  means  and  standard  deviations  came  from  the  scores  obtained  by 
the  58  LCAC  fleet  navigators  in  the  aforementioned  concurrent  validation  study.  The  means  and  standard 
deviations  used  in  this  standardizing  process  are  listed  below  in  Table  1. 


Table  1:  The  means  and  standard  deviations  on  each  of  the  eight  test  scores  as  calculated  from  the  58 
LCAC  fleet  navigators.  These  means  and  standard  deviations  were  used  to  standardize  subsequent  scores  as 
obtained  from  the  LCAC  candidates. 


Number 

Test 

Mean 

Standard  Deviation 

i 

Absolute  Difference 

38.770 

16.76 

2 

Percentage  Correct  on  CVT 

0.716 

0.08 

3 

RT  to  CVT  Questions 

11.202 

3.15 

4 

RT  to  CVT  Answers 

8.094 

1.70 

5 

Dichotic  Listening  Task 

102.121 

5.04 

6 

Manikin 

51.678 

20.77 

7 

Dual  Task  (Stick  with  DLT) 

19.234 

3.36 

8 

Time  Estimation  Task 

3.330 

0.22 

The  absolute  difference  score  was  formed  by  taking  the  number  correct  minus  the  number  incorrect  as  averaged 
over  two  sessions.  The  Dual  Task  score  was  formed  by  taking  the  DLT  score  and  dividing  by  the  logarithm  to  the 
base  10  of  the  tracking  error  score  using  the  stick.  The  Manikin  score,  like  the  Absolute  Difference  score,  was 
number  correct  minus  number  incorrect  averaged  over  two  sessions.  The  Time  Estimation  score  was  the  average  of 
the  logarithm  to  the  base  10  of  the  tracking  error  portion  of  the  test  over  two  sessions. 

For  three  of  these  scores,  1)  RT  to  answers  on  CVT,  2)  RT  to  questions  on  CVT,  and  3)  Time  Estimation  Task, 
the  standardized  score  as  computed  above  was  made  negative  if  positive  and  likewise  made  positive  if  negative. 
The  reason  for  this  flip-flop  is  because  lower  RT  and  lower  tracking  error  represent  better  performance  while 
higher  RT  and  higher  tracking  error  represent  worse  performance. 

THE  BAYESIAN  FRAMEWORK  FOR  THE  PREDICTION  ALGORITHM 

The  algorithm  encoded  in  the  system  software  predicts  whether  a  candidate  for  the  LCAC  navigator  position 
will  pass  or  fail  training.  The  algorithm  compares  the  distance  of  a  composite  score  earned  by  a  candidate  to  the 
mean  composite  score  for  a  group  of  trainees  who  have  already  passed  training  and  to  the  mean  composite  score 
for  a  group  of  trainees  who  have  already  failed  training.  If  the  candidate’s  composite  score  is  closer  to  the  pass 
group,  then  the  candidate  is  predicted  to  pass.  Otherwise,  the  candidate’s  score  is  closer  to  the  mean  of  the  fail 
group  and  is  therefore  predicted  to  fail.  The  mathematical  derivation  to  arrive  at  this  algorithm  and  a  couple  of 
numerical  examples  are  provided  to  show  how  it  works  in  practice. 
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A  result  from  Bayesian  statistical  decision  theory  (Blower,  1996)  states  that  if  the  likelihood  ratio  is  greater 
than  or  equal  to  some  number,  /?,  then  a  pass  is  predicted.  If  the  likelihood  ratio  is  less  than  /?,  then  a  fail  is 
predicted.  In  symbols,  _ 

If  £(z)  >  /?,  predict  PASS 
Otherwise,  predict  FAIL 
where  £(x),  the  likelihood  ratio,  is  defined  as 

,  \  __  ^(Composite  Score|Pass) 

P(Composite  Score|Fail) 

/5  is  determined  by  the  ratio  of  the  prior  probabilities  of  pass  and  fail  and  by  the  relative  costs  of  making 
correct  and  incorrect  decisions.  In  this  situation  of  equal  prior  probabilities  and  symmetrical  costs  for  right  and 
wrong  decisions,  (5  is  equal  to  1.  The  prediction  algorithm  nowr  reads  as 

If  £(x)  >  1,  predict  PASS 
Otherwise,  predict  FAIL 


The  only  computation  that  remains  is  the  determination  of  the  likelihood  ratio,  £(x).  The  composite  score  is 
derived  from  a  discriminant  analysis.  The  discriminant  analysis  forms  a  weighted  linear  combination  of  eight 
variables  from  the  test  battery  called  the  composite  score.  The  separation  between  the  mean  of  the  composite 
scores  of  those  that  passed  and  the  mean  of  the  composite  scores  of  those  that  failed  is  made  as  wide  as  possible 
while,  at  the  same  time,  the  spread  of  composite  scores  within  each  of  the  two  groups  is  made  as  narrow  as 
possible.  These  composite  scores  are  generally  assumed  to  be  normally  distributed  and  by  construction  to  have  a 
standard  deviation  of  1.  Therefore,  the  probability  density  function  (pdf)  for  a  composite  score,  x,  is  a  normal  pdf, 


P  (composite  score)  = 

1 

\/2k  a 

exp  j 

1  / X 

2  V 

?>') 

(2) 

For  the  pass  group,  this  is  expressed  as 

P(composite  score|Pass) 

1 

y/2? T 

exp  | 

4(x- 

(3) 

and  for  the  fail  group  as 

^(composite  score|Fail) 

1 

y/2K 

exp  | 

i 

hTcS 

1 

—  /A Fail )  | 

(4) 

since  (Jpass  —  &fq.h  —  L 


Forming  the  likelihood  ratio  of  the  composite  scores  results  in 


C(x) 


P(Composite  Score|Pass) 
P(Composite  Score|Fail) 


•$£;  exp{-^(i-/xpa^)2} 
exp{-i(x-/xFai,)2} 


=  exp 


=  exp 


|  2^X  f^Po-ss)  "F  2 MFai/) 

^  (•£  ~  f^Fail)  (%  ^ 


(5) 

(6) 

(7) 

(8) 
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The  exponential  format  can  be  eliminated  by  taking  the  natural  logarithm  of  both  sides  of  the  prediction  algorithm. 


In  £(x) 

> 

In/? 

(9) 

(x  ~  l^Fail)^  ftp  ass)2 

2 

> 

In  1 

(10) 

(x  *—  /Z  Fail)  ~  {x  —  lip  ass) 

> 

2  In  1 

(11) 

(x  —  fiFail)  —  {x  —  fipass) 

> 

0 

(12) 

(x-fiFail)2 

> 

(x  —  fipass) 

(13) 

The  negative  of  the  squared  terms  could  be  taken  as  well  and,  finally,  since  Equation  (13)  just  represents  a 
distance  along  a  line,  we  can  dispense  with  the  squaring  and  take  the  absolute  value  of  the  two  terms  as  the  final 
form  of  the  algorithm. 

If  |  I^Faii  -  x  |  >  |  fipass  —  x  |,  then  predict  PASS 
Otherwise,  predict  FAIL 

This  illustrates  the  remark  in  the  opening  paragraph  that  a  simple  distance  calculation  to  the  mean  of  the  two 
groups  is  all  that  is  needed  to  determine  the  proper  classification. 

THE  REVISED  ALGORITHM 

This  section  describes  the  revised  prediction  algorithm  put  in  place  as  of  December  1997.  It  reflects  the  data 
from  13  subjects  tested  at  NAMRL  from  May  1996  to  November  1997  for  which  training  outcomes  were  available 
and  5  subjects  tested  at  ACU-4  and  ACU-5  in  October  and  December  of  1995.  These  18  subjects  all  entered 
LCAC  navigator  training,  and  criterion  data  as  to  whether  they  subsequently  passed  or  failed  training  had  been 
recorded.  Five  additional  subjects  were  added  to  help  determine  the  revised  algorithm;  these  were  the  predicted 
failures  from  the  original  algorithm,  which  we  assumed  would  have  been  actual  failures  had  they  entered  training. 
Thus  a  total  of  23  subjects  formed  the  basis  for  the  revised  algorithm. 

The  prediction  algorithm  described  in  this  section  will  be  used  to  screen  LCAC  navigator  candidates 
commencing  in  December  1997  to  some  date  in  the  near  future  when  the  algorithm  will  be  revised  as  more 
candidates  are  tested  and  more  training  outcomes  received.  The  original  algorithm  used  prior  to  this  revised 
algorithm  is  described  in  the  next  section.  This  original  algorithm  screened  the  18  subjects  mentioned  above  plus 
an  additional  12  candidates  who  are  either  currently  in  training  or  awaiting  training. 

A  numerical  example  of  the  prediction  algorithm  contained  in  the  last  outlined  box  follows.  The  coefficients 
for  the  composite  score,  x,  are  found  by  discriminant  analysis.  The  coefficients  for  each  of  the  eight  tests  are  listed 
in  Table  2.  These  are  unstandardized  coefficients,  c;,  which  multiply  the  score,  S',.  obtained  by  a  candidate  on 
each  of  the  eight  tests.  The  composite  score  is  computed  then  as, 

8 

x  =  CiSi  +  Constant  (14) 

i= 1 

For  example,  subject  364  obtained  the  following  scores  as  listed  in  Table  3.  The  mean  of  the  composite  scores 
for  the  FAIL  group  was  UFaii  =  —1.761,  and  the  mean  of  the  composite  scores  for  the  PASS  group  was 
ptpass  —  1.614.  It  is  obvious  that  this  candidate’s  score  is  closer  to  the  mean  of  the  PASS  Group  and,  therefore, 
would  be  classified  as  a  predicted  PASS.  Formally,  the  prediction  algorithm  works  out  as, 

|  fJ'Fail  —  1.897  |  >  I  UPass  ~  1-897  | 


9 


Table  2:  The  coefficients,  c,,  of  the  composite  score  for  the  revised  algorithm  as  determined  by  discriminant 
analysis. 


Number 

Test 

Coefficient 

1 

Absolute  Difference 

2.031 

2 

Percentage  Correct  on  CVT 

0.614 

3 

RT  to  CVT  Questions 

0.138 

4 

RT  to  CVT  Answers 

-0.143 

5 

Dichotic  Listening  Task 

-0.449 

6 

Manikin 

-0.024 

7 

Multi-Task  Stick  with  DLT 

-0.124 

8 

Time  Estimation  Task 

0.709 

9 

Constant 

-1.089 

Table  3:  The  calculation  of  the  composite  score  for  an  LCAC  navigator  candidate  who  is  a  predicted  PASS. 


Number 

Test 

Multiplication 

1 

Absolute  Difference 

0.5507 

2.031 

1.118 

2 

Percentage  Correct  on  CVT 

1.6750 

0.614 

1.028 

3 

RT  to  CVT  Questions 

0.5530 

0.138 

4 

RT  to  CVT  Answers 

-1.0447 

-0.143 

5 

Dichotic  Listening  Task 

1.1665 

-0.449 

-0.524 

6 

Manikin 

-0.6346 

-0.024 

7 

Multi-Task  Stick  with  DLT 

1.1678 

-0.124 

8 

Time  Estimation  Task 

1.7861 

0.709 

1.266 

9 

Constant 

1.0000 

-1.089 

-1.089 

CiSi  +  Constant  =  1.897 


|  -1.761  -  1.897  | 


>  |  1.614-  1.897  | 


3.658  >  .283 


Since  this  is  true,  the  algorithm  predicts  PASS. 

An  example  of  a  predicted  FAIL  according  to  the  revised  algorithm  is  subject  365  with  scores  as  listed  in 
Table  4.  Again,  it  is  immediately  clear  that  this  composite  score  is  closer  to  the  mean  of  the  FALL  group  than  to 
the  mean  of  the  PASS  group  and  thus  would  be  a  predicted  FAIL.  The  formal  computation  as  dictated  by  the 
prediction  algorithm  to  reach  a  similar  conclusion  would  be 

I  UFaii  -  (-3.325)  I  >  I  nPas,  -  (-3.325)  | 

|-1.761  +  3.325|  >  |  1.614  +  3.325 1 

1.564  >  4.939 

Since  this  not  true,  the  algorithm  predicts  a  FAIL. 
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Table  4:  The  calculation  of  the  composite  score  for  an  LCAC  navigator  candidate  who  is  a  predicted  FAIL. 


Number 

Test 

Score 

Coefficient 

Multiplication 

1 

Absolute  Difference 

-1.0304 

2.031 

-2.092 

2 

Percentage  Correct  on  CVT 

0.6333 

0.614 

0.389 

3 

RT  to  CVT  Questions 

-1.4248 

0.138 

-0.197 

4 

RT  to  CVT  Answers 

-1.7212 

-0.143 

0.246 

5 

Dichotic  Listening  Task 

0.7696 

-0.449 

-0.346 

6 

Manikin 

-0.8753 

-0.024 

0.021 

7 

Multi-Task  Stick  with  DLT 

1.0303 

-0.124 

-0.128 

8 

Time  Estimation  Task 

-0.1828 

0.709 

-0.130 

9 

Constant 

1.0000 

-1.089 

-1.089 

Ci^i  +  Constant  =  —3.325 


Note  that  the  original  algorithm  predicted  this  candidate  as  a  PASS  as  explained  in  the  next  section.  The  newly 
revised  algorithm  corrects  this  mistake  made  by  the  original  algorithm  as  well  as  several  other  wrongly  predicted 
PASSes.  The  2x2  classification  table  of  results  for  the  23  LCAC  navigator  candidates  for  which  we  currently 
possess  training  outcomes  is  shown  in  Table  5.  The  revised  algorithm  has  only  two  misclassifications. 


Table  5:  The  classification  table  of  results  for  23  candidates  using  the  revised  algorithm. 


Actual 


Prediction 
PASS  FAIL 


PASS 

FAIL 


11 

1 

1 

10 

12  11 


12 

11 

23 


The  predicted  FAILs  of  the  original  algorithm  are  also  all  predicted  FAILs  under  the  revised  algorithm. 

Because  the  candidates  who  were  predicted  FAILs  under  the  original  algorithm  never  actually  entered  training,  it  is 
an  assumption  that  they  would  have  all,  in  fact,  failed  training.  Indeed,  this  is  an  assumption  that  we  know  is  not 
entirely  correct.  Some  of  the  predicted  FAILs  under  any  algorithm  would  have  successfully  passed  training  5m  in 
this  early  development  of  the  predictive  algorithm,  we  will  consider  all  predicted  FAILs  as  actual  FAILs.  Later  on, 
after  more  data  are  collected,  we  can  develop  models,  where  some  of  the  predicted  FAILs  who  had  the  highest 
probability  of  actually  passing,  could  be  treated  as  actual  PASSes.  These  new  models  would  average  over  the 
uncertainty  surrounding  these  ambiguous  cases. 

THE  ORIGINAL  ALGORITHM 

Because  the  sponsor  wanted  to  make  the  decision  to  admit  LCAC  navigator  candidates  into  training  without 
conducting  a  study  where  criterion  data  were  gathered  first,  we  had  to  guess  at  an  initial  model.  We  were  not 
entirely  in  the  dark,  however,  because  a  concurrent  validation  study  had  been  run  on  LCAC  navigators  already  in 
the  fleet.  These  58  subjects  took  the  same  battery  of  tests  as  did  the  LCAC  candidates  at  their  respective  fleet 
squadrons,  ACU-4  and  ACU-5,  in  October  and  December  of  1995.  By  definition,  these  subjects  foimed  a 
distribution  of  composite  scores  from  the  PASS  group.  What  was  missing  was  a  comparable  set  of  scores  from 
subjects  who  had  failed  training.  Without  a  FAIL  group,  no  discriminant  analysis  could  be  performed  to  find  an 
optimal  set  of  coefficients  to  form  a  linear  composite  score  that  best  separated  PASSes  from  FAILs. 
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The  initial  guess  at  a  composite  score  was  to  simply  weight  each  test  equally  and  in  a  positive  direction.  These 
composite  scores  were  plotted  for  the  58  fleet  navigators.  The  mean  of  these  composite  scores  for  this  PASS  group 
equaled  fipass  =  0,  with  a  standard  deviation  of  apQSs  —  -52.  To  choose  a  cut-off  score,  we  decided  to  place  the 
cut-off  score  at  the  bottom  20%  of  this  distribution.  This  resulted  in  a  threshold  score  of  -.375. 

The  calculation  of  the  composite  scores  under  the  original  algorithm  are  repeated  for  subjects  364  and  365  and 
shown  in  Tables  6  and  7,  respectively.  Both  composite  scores  are  above  -.375  and  therefore  both  candidates  were 
predicted  PASSes.  This  was  an  incorrect  decision  for  subject  365  as  he  eventually  failed  training.  Indeed,  the 
original  algorithm  did  not  work  very  well  as  5  of  the  13  predicted  PASSes  eventually  failed  training. 


Table  6:  The  calculation  of  the  composite  score  under  the  original  algorithm  for  an  LCAC  navigator  candidate 
(#364)  who  is  a  predicted  PASS. 


Number 

Test 

Score 

Coefficient 

Multiplication 

i 

Absolute  Difference 

0.5507 

0.125 

0.069 

2 

Percentage  Correct  on  CVT 

1.6750 

0.125 

0.209 

3 

RT  to  CVT  Questions 

0.5530 

0.125 

0.069 

4 

RT  to  CVT  Answers 

-1.0447 

0.125 

-0.131 

5 

Dichotic  Listening  Task 

1.1665 

0.125 

0.146 

6 

Manikin 

-0.6346 

0.125 

-0.079 

7 

Dual  Task  (Stick  with  DLT) 

1.1678 

0.125 

0.146 

8 

Time  Estimation  Task 

1.7861 

0.125 

0.223 

=  0.6525 


Table  7:  The  calculation  of  the  composite  score  under  the  original  algorithm  for  an  LCAC  navigator  candidate 
(#365)  who  is  a  predicted  PASS. 


Number 

Test 

Score 

Coefficient 

Multiplication 

1 

Absolute  Difference 

-1.0304 

0.125 

-0.129 

Percentage  Correct  on  CVT 

0.6333 

0.125 

0.079 

RT  to  CVT  Questions 

-1.4248 

0.125 

-0.178 

RT  to  CVT  Answers 

-1.7212 

0.125 

-0.215 

Dichotic  Listening  Task 

0.7696 

0.125 

0.096 

6 

Manikin 

-0.8753 

0.125 

-0.109 

7 

Dual  Task  (Stick  with  DLT) 

1.0303 

0.125 

0.129 

8 

Time  Estimation  Task 

-0.1828 

0.125 

-0.023 

; x>Si =  -o.35o 


As  mentioned  previously,  this  algorithm  was  in  effect  to  screen  LCAC  candidates  from  the  beginning  of  testing 
in  May  1996  until  the  revised  algorithm  went  into  effect  in  December  1997.  A  total  of  30  LCAC  candidates  were 
screened  by  the  original  algorithm;  25  candidates  were  predicted  PASSes  under  this  algorithm  and  entered  LCAC 
navigator  training.  Five  candidates  were  rejected  from  entry  into  training  because  their  composite  scores  fell  below 
the  threshold  of  -.375.  These  5  candidates  would  also  have  been  predicted  FAILs  under  the  revised  algorithm. 
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DISCUSSION 


Our  goal  was  to  develop  a  computer-based  selection  system  that  was  tailored  specifically  to  the  LCAC 
navigator  position,  a  community  that  had  experienced  attrition  rates  in  excess  of  50%.  NAMRL  had  successfully 
transitioned  a  previous  selection  system  for  the  LCAC  operators  and  engineer’s  programs  using  a  predictive 
validation  methodology.  For  this  system,  however,  the  program  sponsor  wanted  a  preliminary  model  for  screening 
navigators  as  quickly  as  possible.  So,  based  on  the  information  gathered  from  a  concurrent  validation  sample,  a 
preliminary  model  was  generated,  and  a  composite  score  was  computed  from  each  of  the  subtests.  The  decision 
was  then  made  by  the  sponsor  to  set  the  cut-off  score  at  the  20th  percentile  of  the  concurrent  validation  sample. 

Thirty  LCAC  navigator  candidates  have  been  screened  by  the  preliminary  algorithm.  Five  of  these  were 
predicted  failures  and  therefore  did  not  enter  training.  Of  the  remaining  25  who  were  predicted  passes,  13  have 
have  completed  training,  and  we  are  awaiting  the  training  outcome  of  the  remaining  12.  The  23  candidates 
represented  in  Table  5  consist  of  the  13  candidates  who  have  completed  training,  5  predicted  failures  for  whom  we 
assumed  actual  failure,  and  5  students  tested  at  ACU-4  and  ACU-5.  Of  the  13  candidates  who  completed  training 
and  were  screened  by  the  original  algorithm  as  predicted  passes,  5  have  failed  training.  This  prompted  us  to  seek 
an  early  modification  to  the  original  predicition  algorithm. 

This  report  presents  the  first  modification  of  the  preliminary  model  using  a  predictive  validation  sample.  The 
new  model,  which  should  improve  the  ability  of  the  system  to  screen  those  applicants  most  likely  to  succeed  in 
training  accurately  predicts  tire  training  outcome  for  98%  of  the  cases/applicants.  The  major  difference  between 
these  models  is  the  increased  weightings  for  some  of  the  subtests.  Overall,  tests  that  included  a  psychomotor 
tracking  (stick)  component  were  not  as  predictive  of  eventual  success  in  navigator  training  as  the  other  tests.  The 
AD,  CVT,  and  TET  are  most  heavily  weighted  in  the  new  algorithm,  with  better  performance  -  measured  by  the 
percentage  of  correct  responses  for  the  AD  and  CVT  task  and  pixel  distance  from  target  for  the  TET  --  being 
associated  with  a  higher  probability  of  success  in  training.  These  findings  are  not  that  surprising  given  that  the 
primary  tasks  of  a  navigator  involve  visual  recognition,  time/course  corrections  and  computations,  as  well  as 
time/distance  observations  based  upon  images  on  their  CRTs.  These  tasks  match  up  relatively  well  with  the  basic 
skills  measured  by  each  of  these  subtests,  however,  the  findings  for  two  of  the  subtests,  the  DLT  and  Manikin  do 
not  make  intuitive  sense.  These  tests  have  been  found  to  be  predictive  of  other  criterion  in  this  laboratory,  both 
within  the  LCAC  and  aviation  communities  (Blower  &  Dolgin,  1991;  Dolgin,  et  al.,  1992).  However,  in  the 
current  study,  both  tests  have  relatively  smaller  weightings  in  the  final  model  than  the  other  variables,  and  the 
direction  of  the  relationships  (better  performance  predicting  training  failure)  are  opposite  to  those  that  were 
expected.  As  greater  numbers  of  applicants  are  screened  using  the  new  system  model  and  more  criterion  data 
become  available,  it  is  possible  that  these  unexpected  relationships  will  stabilize,  change  direction  or  disappear 
entirely. 
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APPENDIX  A.  EXAMINER’S  MANUAL  FOR  THE  LCAC 
NAVIGATOR  SELECTION  SYSTEM 


GETTING  THE  SYSTEM  READY 

1 .  Turn  on  the  computer. 

2.  It  should  automatically  start  in  Windows.  However,  if  it  the  C:\  prompt  appears  on  the  screen,  type  WIN 
and  hit  enter. 

TO  CALIBRATE  THE  JOYSTICKS 

3 .  Once  in  windows,  double  click  on  the  icon  titled  “MAIN.” 

4.  Then  double  click  on  “CONTROL  PANEL.” 

5.  Then  double  click  on  “JOYSTICKS.” 

6.  On  the  screen  you  will  see  that  the  button  next  to  the  label  “JOY STICK  1  ”  should  be  highlighted. 

“Joystick  1  ”  is  the  joystick  that  is  in  the  center  of  the  system  as  you  are  sitting  facing  the  monitor.  It  moves  in 
all  four  directions  (up,  down,  left,  right).  Click  on  the  button  on  the  screen  that  says  “CALIBRATE.” 

7.  To  calibrate  Joystick  1  follow  the  directions  that  are  given  on  the  screen.  First  move  Joystick  1  to  the 
upper  left  position  and  click  the  trigger  button. 

8.  Now  move  Joystick  1  to  the  lower  left  position  and  again  click  on  the  trigger  button. 

9.  Now  move  it  to  the  lower  right  position  and  click  the  trigger  button. 

1 0.  Finally,  move  it  to  the  upper  right  position  and  click  the  trigger  button. 

1 1 .  Now  check  that  the  calibration  is  centered.  Do  this  by  clicking  the  button  that  says  “TEST”.  The  cross 
hair  should  be  centered  on  the  screen.  If  it  is  not  centered  use  the  black  dials  located  on  the  pad  of  Joystick  1  to 
center  the  cross  hair  as  much  as  possible.  You  may  not  be  able  to  get  it  perfectly  centered,  but  get  it  as  close  as 
possible.  When  you  are  done  centering  the  cross  hair,  click  the  “DONE”  button.  If  had  to  mess  with  the  dials, 
then  it  is  VERY  IMPORTANT  that  you  RE-CALIBRATE  the  joystick  again.  (Start  back  at  #  6.)  If  you 
didn’t  move  the  dials,  then  proceed  with  #  12. 

1 2.  Once  you  have  completed  the  calibration  of  Joystick  1 ,  you  will  now  have  the  option  to  choose  Joystick  2. 
Click  the  button  to  the  right  of  this  title  (on  the  screen). 

13.  Joystick  2  is  a  combination  of  the  stick  (located  to  your  left  side  as  you  face  the  computer)  and  the  rudder 
pedals  (located  at  your  feet). 

14.  The  stick  moves  up  and  down,  while  the  rudder  pedals  control  left  and  right  motion. 

15.  Now  choose  to  “CALIBRATE.” 

1 6.  Again,  follow  the  instruction  provided  on  the  screen.  First  move  the  stick  in  the  up  position  and  the  move 
the  rudder  pedal  into  the  left  position.  Do  this  by  moving  your  left  leg  forward.  Once  this  is  done,  click  the 
trigger  button  on  the  stick. 
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17.  Leaving  the  rudder  pedals  in  the  left  position,  pull  the  stick  into  the  down  position  and  click  the  trigger 
button. 

1 8.  Now,  leaving  the  stick  in  the  down  position,  move  the  rudder  pedals  into  the  right  position.  This  done  by 
moving  your  right  leg  all  the  way  up.  Once  this  is  done,  click  the  trigger  button. 

1 9.  Finally,  leaving  the  rudder  pedals  in  the  right  position,  move  the  stick  into  the  up  position  and  click  the 
trigger  button. 

20.  Now  check  that  the  calibration  is  centered.  Do  this  by  clicking  the  button  that  says  “TEST.  The  cross 
hair  should  be  centered  on  the  screen.  If  not,  check  two  different  things  First,  it  is  off  center  in  the  up  and 
down  position?  If  yes,  then  move  the  black  dial  on  Joystick  2  until  the  cross  hair  is  centered.  Second,  if  it  is  off 
center  in  the  left  and  right  position  then  you  have  to  rotate  the  dial  located  on  the  rudder  pedals.  This  dial  is 
found  in  between  the  two  pedals.  It  looks  more  like  a  cog  wheel.  You  have  to  pull  up  on  it,  then  rotate  it  until 
the  cross  hair  is  centered.  Once  you  get  it  as  close  to  center  as  you  can,  you  have  to  push  it  back  down  into 
place  again.*  You  may  not  be  able  to  get  it  perfectly  centered,  but  get  it  as  close  as  possible.  When  you  have 
finished  with  the  test,  click  the  “DONE”  button.  If  you  had  to  adjust  the  dials,  then  it  is  VERY  IMPORTANT 
that  you  RECALIBRATE  the  joystick  again.  (Start  back  at  #  12.)  If  you  didn’t  move  the  dials,  then  proceed 

with  #  21. 


*  It  is  also  important  that  you  ALWAYS  make  sure  that  the  cogs  are  flush  with  one  another,  or  you  can  lose  calibration 
during  testing. 


After  the  system  is  calibrated,  click  the  OK  button  and  the  computer  will  save  the  calibrations. 

TO  START  TESTING 

1 .  You  should  still  be  in  the  Windows  screen. 

2.  To  start  the  LCAC  test,  double  click  on  the  icon  that  reads  “LCAC  NAVIGATOR.” 

3.  The  next  screen  will  be  a  picture  of  the  LCAC  with  the  following  options:  TEST,  DEMO,  and 

“EXIT.” 

INSTRUCTIONS  TO  EXAMINEES: 

Say: 

“This  is  a  test  that  measures  eye-hand  coordination,  cognitive  abilities,  spatial  abilities,  and  listening  skills. 

Show  them  the  system  and  say: 

"This  is  the  station  that  you  will  be  working  at.  You  will  only  be  using  the  joystick  in  the  center,  the  headphones,  and 
the  number  keypad.  When  you  hold  the  joystick,  hold  it  like  you  would  normally  hold  a  joystick  with  your  index  finger 
on  the  trigger  button.  For  most  of  the  tests  you  will  not  be  using  this  button;  however,  we  have  found  that  it  helps  you 
to  stabilize  your  wrist  and  gives  you  more  control  over  the  object.  This  joystick  moves  in  all  four  directions. 

Notice  the  black  dials  on  the  gray  part  of  the  joystick  It  is  important  that  throughout  testing  you  Do  Not  play  with 
these  dials  in  any  fashion.  They  are  part  of  the  calibration  and  by  changing  them,  you  can  damage  your  scores. 
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Show  them  the  headphones  and  then  say: 

“Look  at  these  headphones .  You  will  only  be  using  these  headphones  for  two  of  the  subtests ,  so  you  will  not  need  to 
wear  them  throughout  the  entire  test.  The  computer  will  tell  you  when  you  need  to  put  them  on.  Notice  that  one  side  is 
labeled  left  and  one  side  is  labeled  right.  It  is  important  for  you  to  put  the  left  on  the  left  ear  and  the  right  on  the  right 
ear.  The  reason  for  this  is  that  you  will  be  doing  a  dichotic  listening  test.  In  this  test ,  information  will  be  presented  to 
you  through  the  headphones.  Each  ear  will  receive  different  information  in  the  form  of  numbers  and  letters.  The 
computer  will  tell  which  ear  you  should  listen  to.  For  instance,  the  computer  will  say,  Test  I,  Left  Ear.  At  this  point 
you  need  to  ignore  the  information  coming  into  the  right  ear  and  input  only  the  numbers  you  hear  coming  into  the 
left  ear.  You  need  to  enter  these  numbers  into  the  keypad  as  you  hear  them.  Do  Not  try  to  memorize  the  numbers  and 
then  enter  them.  You  will  not  get  proper  credit  for  your  responses.  The  first  time  that  you  use  the  headphones,  it  will 
be  just  these  responses  alone  that  you  will  be  making..  You  may  be  tempted  to  use  your  right  hand  to  enter  the 
numbers.  However ;  you  will  need  to  use  your  left  hand  because  the  next  time  you  use  the  headphones  you  will  be 
using  your  right  hand  to  control  the  stick  and  keep  an  object  centered  on  the  screen.  So,  use  your  left  hand  when 
entering  the  numbers  into  the  keypad.  ” 

“You  will  get  a  practice  session  before  some  of  the  subtests  begin.  For  those  that  you  do  not,  I  am  going  to  explain 
them  to  you  and  then  demo  them  for  you.  ” 

You  will  only  demo  three  subtests:  The  Absolute  Difference  Task,  the  Manikin,  and  the  Time  Estimation  Task. 

At  this  time,  sit  down  at  the  test  station  and  click  on  the  icon  that  says  “DEMO.”  Change  the  demo  time  to  .5.  Do 
that  by  moving  the  mouse  to  the  area  that  says  “DURATION,”  highlight  the  “1,”  and  type  in  the  appropriate  time,  “.5.” 

“The  first  test  that  Vm  going  to  show  you  is  called  the  Absolute  Difference  Test.  A  series  of  numbers  are  presented  to 
you  on  the  screen  one  at  a  time.  You  are  to  take  the  absolute  difference  between  the  previous  number  presented  and 
the  current  number  on  the  screen.  For  instance,  say  that  the  #  8  is  the  first  number  presented,  then  the  #  7.  You 
should  enter  the  absolute  difference  between  the  S  and  the  7  into  the  keypad,  which  is  1.  Once  you  do  that,  a  new 
number  will  appear.  Let ’s  say  it  is  the  #  9.  Now  you  will  take  the  absolute  difference  between  the  #  7,  which  was  the 
previous  number,  and  the  #  9.  You  then  need  to  enter  that  number,  which  is  2,  into  the  keypad. 

At  this  point,  you  should  start  the  demonstration.  Click  on  the  icon  labeled  “ABSOLUTE  DIFFERENCE.”  You 
need  to  explain  to  the  subjects  that  when  the  instructions  say  hit  the  #1  key  that  it  is  only  to  start  the  program  and  has  no 
other  significance. 

Once  you  have  completed  this  demo,  move  on  to  the  next  subtest. 

Say: 

“The  next  test  that  I  m  going  to  show  is  called  the  Manikin.  You  will  see  the  image  of  a  sailor.  In  one  hand  he  is 
holding  a  green  circle,  in  the  other  hand  he  is  holding  a  red  square.  Your  task  is  to  determine  which  hand  is  holding 
the  red  square.  It  is  important  that  you  remember  this  because  once  the  test  begins,  you  will  not  be  prompted  with 
that  information  again.  The  sailor  can  be  in  four  different  positions.  He  can  be  right-side  up  and facing  you,  right- 
side  up  and  backwards,  upside-down  and  facing  you,  or  upside-down  and  backwards.  You  will  be  using  the  #  1  key  to 
choose  the  left  hand  and  the  #  2  key  to  choose  the  right  hand.  ” 

Start  the  demo  and  as  you  choose  which  hand  is  holding  the  square,  say  aloud  the  proper  hand.  Ask  them  if  they 
have  any  questions.  Then  move  to  the  next  subtest. 

Say: 
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“The  next  test  is  called  the  Time  Estimation  Task  This  is  a  dual  tasking  test,  which  simply  means  you  will  be  doing 
two  things  at  once.  The  first  thing  that  you  will  be  doing  is  controlling  a  cross  hair  at  the  bottom  of  the  screen  using 
the  joystick  in  front  of  you.  At  the  top  of  the  screen,  an  airplane  will  appear  on  the  left  hand  side,  only  staying  visible 
for  about  2  seconds.  It  will  then  disappear  into  a  cloud.  On  the  right  hand  side  of  the  screen  is  a  missile.  The  object 
of  the  task  is  to  estimate  at  what  point  you  think  the  airplane  will  be  over  the  missile.  At  that  point  you  will  use  the 
trigger  button  to  fire  a  missile.  Once  you  fire,  a  number  will  appear.  A  positive  number  means  that  you  fired  too 
soon  and  a  negative  number  means  that  you  fired  too  late.  A  zero  means  that  you  hit  it  perfectly.  The  speed  of  the 
airplane  will  vary  between  trails,  but  will  remain  constant  within  atrial.  So,  if  initially  it  is  going  at  a  slow  speed,  it 
will  maintain  that  same  slow  speed  throughout  that  trial.  Again,  remember  that  the  whole  time  you  are  doing  this,  you 
should  be  controlling  the  cross  hair  at  the  bottom  of  the  screen. 

At  this  point,  demo  the  subtest  while  repeating  the  task  instructions. 

When  you  have  finished  going  through  all  the  demos,  it  is  time  to  start  the  test.  Exit  the  demo  mode  by  clicking  on 
the  “EXIT”  button.  You  should  be  at  the  main  screen  that  had  the  initial  three  options:  “TEST,”  ‘  DEMO,  EXIT. 
Click  on  the  “TEST”  button.  At  this  point,  an  instruction  screen  will  appear. 


Say: 

“You  are  now  ready  to  begin  the  test.  It  will  take  you  approximately  1  hour  to  take  this  test.  The  computer  will  not 
move  into  the  next  subtest  or  to  another  screen  without  you  physically  doing  something,  either  by  pressing  the  page 
down/page  up  or  the  enter  key.  Therefore,  if  you  need  to  take  a  break  (use  the  restroom  or  get  some  water),  go  ahead. 
If  you  have  any  questions  about  arty  instructions  throughout  the  test  please  come  and  ask  me  before  you  begin  the 
test.  Once  you  start  a  test  I  cannot  stop  it.  Once  you  get  started,  the  computer  will  prompt  you  for  a  Subject  ID  #, 
enter  the  number  that  was  assigned  to  you  earlier.  Do  you  have  any  questions?  Go  ahead  and  begin  the  test.  ” 

At  this  point,  leave  the  individual  to  him-  or  herself. 


ADMINISTERING  THE  CVT 

Once  the  individual  has  completed  the  psychomotor  portion  of  the  test,  you  will  need  to  administer  the  *  Complex 
Visual  Task”  test. 

You  will  be  giving  the  demonstration  on  the  psychomotor  test  station  and  the  actual  test  will  be  given  on  Test 
Station  #  3  (This  is  in  the  comer,  next  to  the  psychomotor  station.)  Double  click  on  the  icon  “CVT.” 

Say: 

“This  test  is  called  the  Complex  Visual  Task  There  are  60  questions  and  it  will  take  you  about  30  minutes  to 
complete.  In  this  test  you  will  be  presented  with  a  series  questions  which  will  be  displayed  on  the  screen  one  at  a  time. 
After  each  individual  question,  you  will  be  shown  a  graphic  based  on  that  question.  You  need  to  examine  the  graphic 
and  determine  the  correct  answer.  Enter  your  answer  using  the  number  keypad  and  then  hitting  the  enter  key.  Once 
you  hit  enter,  you  will  not  be  able  to  change  your  answer.  The  computer  will  provide  you  feedback  as  to  how  you  did 
and  what  the  correct  answer  was.  ” 

At  this  time  begin  the  demo.  You  should  be  at  the  CVT  screen.  You  will  be  given  two  options,  “TAKE  TEST”  or 
“EXIT.”  Choose  “TAKE  TEST”  and  enter  a  bogus  Subject  ID  #. 

Go  past  the  instructions.  Say: 

“lam  not  going  to  go  through  the  instruction  screens  verbatim.  When  you  sit  down  to  take  the  test,  you  have  all  the 
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time  you  need  to  take  to  read  through  the  instructions .  ” 

Show  them  the  screen  that  indicates  the  center  of  the  circle  is  the  red  dot.  Say: 

“You  may  he  asked  a  questions  that  says,  How  many  objects  are  going  through  the  center  of  the  circle?  That  means 
any  object  that  if  it  was  in  motion  would  pass  through  this  red  dot.  ” 

Show  them  the  screen  that  demonstrates  North,  South,  East,  West.  Point  this  screen  out  to  them. 

Flip  through  the  next  few  screens,  briefly  summarizing  what  is  on  them.  Say: 

“The  objects  on  the  sides  come  in  three  sizes:  large ,  medium,  and  small  and  in  three  colors:  red,  green,  and  white :  ” 

Show  them  the  first  example.  Read  the  question  and  then  continue. 

Read  the  second  example  and  continue. 

Finally,  read  the  third  example  and  then  continue  on  to  the  next  screen. 

Say: 

“Some  of  the  questions  will  require  a  true  or  false  answer.  Use  the  keypad  to  enter  1  if  the  answer  of  True  and  2  if  the 
answer  is  False.  Once  you  make  your  response,  you  must  hit  the  Enter  key.  Some  questions  will  require  to  make  a 
number  response :  If  the  correct  answer  requires  a  two-digit  number,  like  12,  press  the  1  key  and  2  key,  then  hit  enter. 
You  will  need  to  remember  that  the  different  shapes  represent  different  objects.  That  information  will  only  be  given  in 
the  instructions.  Try  complete  the  test  as  fast  as  you  can,  but  be  as  accurate  as  possible.  Do  you  have  any  questions? 
Once  you  begin  the  test  itself  I  cannot  answer  any  of  your  questions.  ” 

At  this  point,  sit  the  individual  down  at  the  test  station  to  begin  the  test.  You  will  need  to  set  up  the  test  on  this 
computer. 

1.  Turn  on  the  computer. 

2.  It  should  be  in  Windows,  if  not  type  WIN  at  the  C:\  on  the  screen. 

3.  Once  in  Windows,  double  click  on  the  icon  “TEST  ” 

4.  Now  double  click  on  “CVT.” 

5.  You  should  be  at  the  CVT  screen.  You  will  be  given  two  options,  “TAKE  TEST”  or  “EXIT.”  Choose 
“TAKE  TEST  ” 

1 .  The  subject  needs  to  enter  the  same  Subject  ID  #  that  was  used  on  the  psychomotor  test. 

BACKING  UP  DATA 

1 .  Insert  a  disk  into  the  A:  drive. 

2.  In  the  Windows  screen,  double  click  “MAIN.” 

3 .  Double  click  on  “FILE  MANAGER.” 
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4.  Now  scroll  through  the  directories  until  you  find  LNAV  directory. 

5.  Click  on  this  directory.  You  should  now  see  the  directory'  SCORES. 

6.  Click  on  the  SCORES  directory.  You  will  now  see  all  the  subjects’  files. 

7.  Scroll  through  the  files  until  you  find  the  files  for  that  day’s  testing. 

8.  Select  the  file  that  you  want  to  copy  by  clicking  the  file  once. 

9.  Now  hit  the  F8  key.  It  will  ask  you  where  you  want  to  copy  the  file.  Type  A:  then  hit  enter. 

10.  Once  you  have  copied  all  the  appropriate  files,  exit  the  File  Manager. 

11.  In  the  menu  go  into  File,  then  Exit. 

***  To  back  up  the  CVT  data,  follow  the  same  procedures  as  up.  Look  for  the  CVT  directory. 

TROUBLE  SHOOTING 

If  the  program  should  lose  calibration.  Do  the  following. 

1  Restart  the  computer.  Either  by  turning  it  off  and  then  on  again.  Or  by  hitting  the  “Reset  button  on  the 
computer. 

2.  Go  to  Calibrating  the  Joysticks  and  follow  the  steps  outlined  in  that  section. 

If  the  computer  iust  freezes  up.  Do  the  following. 

1  Restart  the  computer.  Either  by  turning  it  off  and  then  on  again.  Or  by  hitting  the  ‘  Reset  button  on  the 
computer. 

2.  Go  to  Calibrating  the  Joysticks  and  follow  the  steps  outlined  in  that  section. 

PICKING  UP  WHERE  THE  INDIVIDUAL  LEFT  OFF 

Once  you  have  re-calibrated  the  joysticks,  you  need  to  find  out  the  last  subtest  that  the  individual  completed.  To 
find  out,  do  the  following  steps. 

1 .  You  should  still  be  in  Windows.  Double  click  on  the  icon  titled  “MAIN .” 

2.  Then  double  click  on  “FILE  MANAGER.” 

3 .  Now  find  the  LNAV  directory  and  double  click  on  it. 

4.  Once  in  the  LNAV  directory,  double  click  on  the  SCORES  directory. 

5.  Scroll  through  the  directory  until  you  find  the  subject’s  file.  (This  should  be  his/her  subject  id  #.) 

6.  Highlight  this  file  and  then  move  the  cursor  to  the  menu  and  click  on  “FILE.” 

7.  In  the  File  menu,  click  on  “RUN.” 
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8.  The  file  name  will  appear  and  in  front  of  it  type  WRITE  (leave  a  space  between  it  and  the  file  name).  Hit 
Enter. 

9.  The  computer  will  ask  you  if  you  want  to  convert  the  file.  Say  NO. 

1 0.  The  file  that  has  been  selected  will  then  be  open.  Scroll  through  the  file  to  see  which  subtest  is  the  last  one 
completed. 

1 1 .  Each  test  will  have  a  Test  ID  #,  write  the  Test  ID  #  for  the  last  completed  subtest. 

1.  Once  you  have  done  this,  exit  the  file.  It  may  ask  you  to  save  changes.  Say  NO. 

TO  EDIT  THE  TEST  FILE 

13.  You  should  still  be  in  the  LNAV  directory. 

14.  Find  the  “LNAV.CFG”  file. 

15.  Open  this  file  as  in  steps  6-9. 

16.  Once  the  file  is  open,  go  to  the  file  menu  and  select  SAVE  AS. 

17.  Change  the  title  to  “SUBJECT  ID  #.CFG  ” 

1 8.  Once  you  have  done  this  it  is  time  to  edit  the  file. 

1 9.  Delete  the  tests  that  have  already  been  taken.  For  instance,  if  the  person  got  through  TEST  ID  3,  then  you 
delete  the  first  3  tests. 

20.  Once  you  have  modified  the  file,  save  the  changes  and  exit. 

2 1 .  Open  the  “LNAV.EXE”  file  by  double  clicking  on  it. 

22.  Open  “ADMIN.”  Here  you  will  see  a  list  of  directories. 

23.  You  now  need  to  edit  the  Test  Script  File  by  replacing  the  “LNAV.CFG”  file,  with  the  new  file  that  was 
created.  This  is  the  “SUBJECT  ID  #.CFG”  file. 

24.  Click  “SAVE”  and  exit. 

25.  Run  the  test  using  the  same  Subject  ID  #.  The  scores  will  be  appended  to  the  existing  file. 

****  IT  IS  VERY  IMPORTANT  THAT  WHEN  THIS  PERSON  COMPLETES  TESTING  THAT  YOU  RE¬ 
ENTER  THE  ADMINISTRATION  DIRECTORIES  AND  CHANGE  THE  TEST  SCRIPT  FILE  BACK  TO 
“LNAV.CFG”.  REMEMBER  TO  SAVE  THE  CHANGES  ***** 
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